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r^ Abstract. Two geometric interpretations of the bar automorphism in the 

positive part of a quantized enveloping algebra are given. The first is in terms 
of numbers of rational points over finite fields of quiver analogues of orbital 
varieties; the second is in terms of a duality of constructible functions provided 
by preprojective varieties of quivers. 
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1. Introduction 



The canonical basis B of the positive part Uy{Qy of the quantized enveloping 
algebra of a semisimple Lie algebra q, constructed by G. Lusztig |3j, has many 
favourable properties, like for example inducing bases in all the finite dimensional 
irreducible representations of q simultaneously. 

The basis B can be characterized algebraically by its elements being fixed under 
the so-called bar automorphism of [/^(g)"'", and by admitting a unitriangular base 
^v^ . change to the PBW-type bases. 

\^ ' The Hall algebra approach to quantum groups |11| provides a realization of 

Gp . certain specializations Uq{g)'^ via a convolution product for constructible functions 

^^^ ' on varieties i?d parametrizing representations of Dynkin quivers. In this realization, 

the elements of PBW-type bases correspond to characteristic functions of orbits 
i-G ' in Rd under a natural action of an algebraic group G^, whereas the elements of 

C^ . B correspond to constructible functions naturally associated to the intersection 

cohomology complexes of the closures of Gd-orbits. 

It is therefore natural to also ask for interpretations of the bar automorphism in 
terms of the geometry of the varieties Rd, since this automorphism plays a central 
role in defining the canonical basis algebraically. 
$-H ' In the present paper, two such interpretations are given. In the geometric setup 

of [H] , analogues of orbital varieties in the quiver context are constructed. These pa- 
rametrize quiver representations fixing certain fiags, and their numbers of rational 
points over finite fields are shown ('Theorem l3.4|l to give essentially the coefficients 
^M,N of the bar automorphism on a PBW-type basis of Uq{g)'^ . The key ingredi- 
ent in deriving this result in section 3 is a generalization fCorollarv l2.6|l of a very 
useful formula |S] by C. Riedtmann, relating numbers of filtrations of quiver repre- 
sentations over finite fields to cardinalities of orbital varieties; this generalization, 
together with the construction of the orbital varieties, is given in section 2. 

The second interpretation starts from a duality between constructible functions 
on the varieties Rd provided by the preprojective varieties of a quiver, also used 
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by G. Lusztig 0. The coefficients Hm.n are shown ('Proposition 14. 2|l to be essen- 
tiaUy given by a convolution operator derived from a certain twisted version of this 
duahty, constructed in section 4. 

Acknowledgments: The present work was done during a stay of both authors 
at the University of Antwerp in the framework of the Priority Programme " Non- 
Commutative Geometry" of the European Science Foundation. 



2. A GENERALIZATION OF RiEDTMANN'S FORMULA 

2.1. Let fc be a field. We fix a finite quiver Q with set of vertices / and set of arrows 
Qi, whose underlying unoriented graph is a disjoint union of Dynkin diagrams of 
type An, D„, Eq, Ej or Eg,. Fixing a dimension type d = "^^dii G N/, define an 
/-graded vector space Vd := ®i^ik'^^ ■ For any subquotient W of Vd compatible 
with the grading, we set dim (W) :— X]i(<^*"''-W^i)* ^ ^^^ where Wi denotes the 
i-component of W. 

Set Rd := RdiQ) ■■= ®^.,,^jHom(k''',k''^) and Gd := W.GLdM- The alge- 
braic group Gd acts linearly on the affine space Rd by {g.M)a:i^j '■— gjMag~ . 

2.2. Let !^ be a positive integer, and let d = (d-'^ , . . . , d*) be a (/-tuple of elements 
in NI such that d = J2sd'- Let J^d be the set of filtrations i^* = (0 = F'' C . . . C 
F^ C F° = Vd) of the graded space Vd such that dimF"^'^ I F" ^ d",! < s <v. 
The action of the group Gd on Vd provides a transitive action of Gd on Td- 

We fix an arbitrary filtration Fq in Ta- Choosing successive complements, we 
can assume that Vd has a direct sum decomposition Vd ~ ®s=i ^1 (^^ /-graded 
fc-space), such that Fq — 04^^ V^* for s = 1, . . . , i^. This induces a decomposition 
Rd -= ®lt^i Rf by setting Rf = ©^^^^^. Hom{{y^),, (yj),). Let P^ be the sta- 
bilizcr of Fq in Gd- This is a parabolic subgroup of Gd, providing an identification 
^d ^ Gd/Pd- 

We say that an element M in Rd is compatible with a filtration F* in Td if 
Ma{Fl) C F^ for any arrow a : i ^ j in Qi and any s — 1, . . . ,1/. Set 

^d :== {{M,F*), M e Rd, F* e Td, M compatible with F*] d Rd x jcv,. 

The diagonal action of the group Gd on Rd x J-'d respects Xd, and the projections 
Pi and p2 on Rd and J-'d, respectively, are G^-equivariant. 

Set Yd := P2^{Fq), which will be identified with its image Pi(ld) in Rd- We 
have the identification 

(2.1) Gd x^-^ Yd ~ Ad, T^ := {i9P,p~\y), p e Pd} ^ {g-v.gF^). 

Denote by Ud the unipotent radical of Pd and set Xd '■— Gd x'^'* Yd- Let tt be 
the natural projection Ad ^ Gd x^"* Yd — Ad. The Levi decomposition gives 
-Pd = LdUd, where Ld is the Levi of the parabolic Pd- We have a diagonal action 
of Ld on Ad defined by l.{g, y) := {gl~^, l-v), and a left action of Gd on Ad- These 
two actions commute. 

The above direct sum decomposition Vd = 0^ V^ provides a surjection ( : Rd ^ 
Ylg Rd= , which restricts to a surjection C ■ ^d ^ lis Rd^ ■ This clearly defines a 
surjection ( : Gd x^"^ Yd -^ lis Rd" by projecting on the second factor. We obtain 
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the following diagram. 

lis ^d." ^- Xd 

(2.2) It, 

Rd <— -^d -^ ^d — Gd/Pd 

The following lemma follows immediately from the definitions: 

Lemma 2.1. The morphism Q commutes with the action of Ld- The m,orphism,s 
■K , pi and p2 commute with the action of Gd- 

2.3. Let mod /cQ be the category of finite dimensional fc-representations of Q. For 
a representation X in mod kQ of dimension type dim jX) = d, we denote by Ox the 
corresponding Gt^-orbit in Rd- The Gt^-orbits are in bijection with the isoclasses of 
representations of dimension type d in mod kQ by definition. We denote by modfcQ 
this set of isoclasses, and X will be the isoclass corresponding to the representation 
X. 

We now suppose that v is the number of isoclasses of indecomposable repre- 
sentations in modfcQ (which coincides with the number of positive roots of the 
root system corresponding to Q by Gabriel's Theorem), and we denote by Is for 
1 < s < u these indecomposable representations, ordered such that 

(2.3) HomQ{It,Is) = for 1 < s < t < I/, 

where Homg denotes the space of homomorphisms in the category modfcQ (such 
an ordering exists since the category modkQ is directed; for this and other facts 
on modfcQ see, for example, 0). 

We fix two representations N and M in Rd- In the following, we define analogues 
of the varieties introduced in the previous section, naturally associated to N and 
M. 

Let N — (BsNs be the unique decomposition of the representation N such that 
Ns is isomorphic to a direct sum of copies of Is for 1 < s < v. We can suppose 
without loss of generality that the spaces A^^ are compatible with the direct sum 
decomposition Vd = 0^ V^ , so that Ns belongs to i?^''*- 

We define Xj^ as the set of pairs {P,F*) in Xd such that the representation 
induced by P on F^^^/F'^ is isomorphic to Ns for any s = I,..., v. Let X^ 
be the subset of pairs (P, F*) in Xj^ such that P belongs to Om- Set 1^ := 
C~'^iY[s^Ns) C Id. Again, the following lemma follows immediately from the 
definitions: 



Lemma 2.2. Via the identification I^Tzl we have 

Xj^ -Gdx''-' % and X§ ^ Gd x^'^ {Yj^ n Om)- 

We define 

If" ~ Gd x^<^ (IV n Om) C Xj^ := Gd x^<^ % c Xd. 

The left action of Gd and the diagonal action of Ld both stabilize these varieties. 

2.4. From now on, we suppose that fc is a finite field with q elements. For any 
representation P in Rd, we denote by Aut(P) C Gd the stabilizer of P (which 
coincides with the automorphisms of P as an object in mod kQ by definition). With 
the notation of the previous section, let Yn be the fiber of ( over the point (Ns), 



that is, Yn ■■= C^iiNs)) C Yd- Note that Ud acts on Yj 
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Proposition 2.3. With notation as above, we have 

Proof. First we have 

n-'p^\OM) n Xj^ ^ XM ^ Gd x^'^ (Yj^ n Om). 
Since pin commutes with the Gd-action, we obtain 

I -1 -'i-fl\T\r.V I \Gd\-\Yjj-nOM\ 

k Pi [M)nXj^\ - 

From Lemma l2. II we conclude 

\p^\M) nxj^l - 



l^dl • |f/d| 

This imphes the proposition. D 

2.5. In this section, we give a more precise version of Proposition 12. 31 Let Qd '■= 
0j 0[(fc'^') be the Lie algebra of the group Gd- The components of an element ^ in 
Qd will be denoted by fi. Let Ud C Qd be the Lie algebra of Ud- The differential of 
the morphism Gd -^ Gd-N gives rise to a morphism of vector spaces (p : Qd ^ Rd 
given by (t>{£,)a:i^] = £,jNa - N^^i. 

Lemma 2.4. The morphism (j) has the following properties : 

(i) Ker(cp)^EndQ{N), 

(ii) Im{(j}) — Tjv, where Tn '■— Tj^{On) is the tangent space to On at the point N , 

(Hi) Im{(j}) is compatible with the decomposition Rd — ^^ ^ -R^' , and it contains 

the subspace ®^<f Rd , 

(iv) the restriction of (p to Ud is injective. 

Proof, (i) follows from the definition of </> and of the category modfcQ. (ii) is clear. 
The first assertion of (iii) follows from the fact that (p decomposes into a direct sum 
</> = ®s,t4's,t, where 

(2.4) </>,.* : 0ifomfc((l/j^)„ {V^),) -. i?f . 

i 

The second assertion is |3I Lemma 10.4]. To prove (iv), we remark that (/)|ua = 
®s>t 't's,t which imphes that Ker(j)\viA = by formula lOl and (i). D 

We now consider the affine space Ym identified with its tangent vector space 

Tn{Yn) hy x^x + N. 

Proposition 2.5. The space Y/v contains 0(ud). Let E^ be any complement of 
0(ud) in Yat which is compatible with the decomposition ^ Rd of Rd- Then, 
(i) the space En is a direct summand of Tn{On) in Rd, 
(ii) En is a tranversal slice for the action of Ud on Yn ■ 
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Proof. First we remark that, by construction, the space Y/v is compatible with the 
decomposition Rd = ^^ j -R^' . Thus, a complement En as above exists by Lemma 

O(iii)- 

(i) is a consequence of Lemma |^](ii), (iii) and the decomposition [T^ of 0. 

Now we prove (ii). Fix a complement Ejy and let X be in E^- We claim that 
X is the unique element of E^ in the orbit Ud-X. Suppose that Y ~ U.X £ E^, 
with U S Ud- The component of U (induced by the decomposition Vd = ®j, V^) 
belonging to ®iHom{{V^)i, (V^*)i) is denoted by Ug^t for s > t. Note that Us^s is 
the identity for all s = 1, . . . , i^. 

We prove by induction on s — t > that Us,t — 0, for s > t. Fix a pair (s, t) for 
1 < t < s < i^. It is easily seen by a weight argument that the induction hypothesis 
implies that the component Ys^t of F has the following form : Yg.t — Us,tNt—NsUs.t- 
So, we have Us,tNt — NgUs.t — Ys,t G En by the hypothesis on En- This implies 
that Ys^t G En n (/'(ud) = {0}. By Lemma IT^ this gives Us^t = 0. The claim is 
proved. 

In particular, this implies that the action of the group Ud on Yn is free. Thus, 
\En\ — Tjj4 — \YN/Ud\- This equality, together with the claim just proved, gives 
(ii). ' D 

The following corollary can be seen as a generalization of Riedtmann's formula. 

Corollary 2.6. Let En be as in the previous proposition. Then, 

j,j ^_\AutiM)\-\ENnOM\ 



where Tff^ jVi denotes the set of filtrations = M" C . . . C Af "^ C M° = M of 
the representation M with successive suhquotients M"^^ /A'P isomorphic to Ng for 
I <s <iy. 

Proof. By construction, we have J^^^ jy ~ Pi^{M)nXj^. Moreover, as C/d C Gd, 
the previous proposition implies that En n Om is a tranversal slice for the action 
of Ud on Yn fl Om- The corollary then follows from Proposition 12. 31 D 

It is known 10 that there exists a polynomial F^'^ Nii^) ^ ^W' called the 
generalized Hall polynomial, whose value at any prime power q equals the number 
of Fq-rational points of the variety JF^^ ^^ . 

3. Hall algebras and coefficients of the bar automorphism. 

3.1. We define the Euler form < , > on N/ by < d, e >:= J2i£i d-iSi — X^ai-^i ^iej. 
We suppose in this section that A; is a field with q = v^ elements for some v E C The 
dimension type d and the representations M, TV are no longer fixed. For all finite 
sets X on which Gd acts, we denote by Cq^ [X] the set of Grf-invariant functions 
from X to C Define 

H.(Q)= 0CG,[i?rf]. 

The space Hv{Q) is endowed with a structure of N/-gradcd C-algebra by the con- 
volution product : 

{f-g){X) - v<'^'> J2 fiX/U)g{U), f e CaARd], g e CaARe], X e Rd+e, 
ucx 
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where U runs over all subrepresentations of X of dimension type e. It is known [lUj 
that this product defines the structure of an associative algebra on TCy(Q), which 
is called the (twisted) Hall algebra of the quiver Q. 

For any representation M of Q with isoclass IT, let cm = e^ be ydimEndN-dimN 
times the characteristic function of the orbit Om- It is clear that {sjg'ljgg ^od ^n 
is a basis of HviQ)- 

Let Si be the simple representation corresponding to the vertex i in /. It is 
known 11 that there exists an isomorphism rj from the Hall algebra Tlv{Q) to the 
positive part Uq{g)'^ of the quantum enveloping algebra associated to Q, such that 
77 maps esi to the canonical generator e^ of Uq{g)^ . Note that the basis cm = e-jj 
is sent to the so-called Poincare-Birkhoff-Witt basis of Uq{Q)'^ which corresponds 
to a reduced decomposition of the longest Weyl group element naturally associated 
to g, see H 4.12]. 

3.2. We consider the inner product on Hv{Q), called Green form, defined by 

f„ ^ N _ „2dimEndN ^ — If: 

(eM,eN) — V ajyj On,m, 

where S is the Kronecker symbol and ajv/ :— aM{v'^) — |Aut(M)|. Hence, we obtain 
the dual PBW type basis by setting elj = w-^^^-^ndw^^^g^^^ 

The following lemma is an easy consequence of the definition of the convolution 
product in the Hall algebra and of the properties of the decomposition TV = 0^ Ng. 

Lemma 3.1. Suppose that N = ^^N^ is the decomposition of N into powers of 
indecomposables as above. Then, 

(i) Cat = CaTj . . . Cat^ , 

(iij e^ — ejY^ . . -ej^^, 

(iii) ejv„ . . . eN, = Em ^^^"''™''"''*'-P^il'!,....iVi i^^Wi, where 5 = Es dimEndN, + 

Es>t < dimNs,dirnNt >. 

3.3. Using the basis elements bm introduced above, the multiplication in the Hall 
algebra reads as follows: 

\ ^ „.dimEndM+dimEndN+< dim M, dim N>-dimEndX mX /„.2\ „ 

Since the F^ ^ are polynomials, we can thus take the above formula as the defi- 
nition of structure constants for a Q(-;;)-algebra, the generic (twisted) Hall algebra 
|11|. which will still be denoted by Hv{Q)- 

We define a Q-linear involution on Q{v) by u = v^^. We define on Hv{Q) ■ 

- a "-linear involution by e7 = e^ , called the bar involution, 

- the Q(v)-linear antiinvolution a by a{ei) = e^. 

Denote by ujm,n the cat -co efficient of bm in the PBW-basis. It is clear that ujm,n 
is zero if M and N do not have the same dimension type. Following Lusztig j^, we 

use the normahzation nM,N = i;'^»™0"-d™0„^^^^ ^ ^dimEndM-dimEndN ^^^^^^ 

We say that M degenerates to N if Om belongs to the closure of Om- The following 
is proved in 0]. 

Lemma 3.2. For any two representations M , N in vciodkQ, we have: 
(i) if Qm,n y^ 0, then M degenerates to N , 
(a) ^M,M = 1, 
(Hi) flM,N e Z[v^^]. 
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We now want to give a geometric interpretation of the polynomial i}M,N- This 
will be provided by Theorem 13.41 The following proposition precises a result of |31 
Proposition 3.1], where the formula was asserted up to a power of v. 

Proposition 3.3. Let M , N be two representations in modfcQ, and let N = 
^^ Ng be the decomposition into powers of indecompo sables as above. Then, the 
polynomial flM,N £ Z[v'^] is given by 

Us «JV, 



Proof. We first calculate ll>m,n = (gm, e^) by using the adjoint of the bar auto- 
morphism for the Green form. From [HI 1.2.10.], we have : 

From Lemma |0](ii), this gives : 

— _ / ^.\-divaM ,.-< dim N. dim N> /- „ -I'l* > ^('1* ^^ 

^M.N — y-v) V (eM,cr(epj^) . . .cr(e^^)j. 

By 12], the elements e^ belong to the dual canonical basis. Hence, by [3 Lemma 

4.3.], 

for all s = 1, . . . ,iy. We deduce that 

7-; _„—Y^,,f<dtmNs.dimNt>f„ _* _* \ 

^M.N — v ^'5^' (eM,e^^ . . .e^J. 

It remains to calculate the e^^^-component of the product e^ . . . e^ in the dual 
PBW-basis. This is obtained from Lemma Hill : 

where T = —'^^dimEndNs + dimEndM + X)s>t < dim N,. dim Nt > . Now, 
from the interpretation of < , > as the homological Euler form in modkQ, namely 
< dim M. dira N >~ dimHom{M, N) — dimExt^{M, N), we easily obtain 

dim,End{N) = y^^dimEnd{Ns) + ^^ < dimNg, dimNt >, 

s 

and the claimed formula follows. 



s<t 



a 

3.4. Our efforts are rewarded in that we can deduce a geometric interpretation of 
the coefficient ^Im.n- 

Theorem 3.4. Let k be the finite field ¥^2 . Fix a dimension type d in NI , and 
fix representations N, M in Rd. Let En be a graded complementary of the tangent 
space of On at N as in Provosition \2.5\ Then, the value of the polynomial ^m.n 
at v"^ equals the cardinality of the set En D Om ■ 

Proof. This is Proposition 13.31 combined with Corollarv l2.6l D 

Note that the theorem implies the following curious identity : 

Corollary 3.5. Let d be in NI and N in Rd. Then, 

p 

where P runs over the set of isoclasses of representations of dimension type d. 
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We finish the section with the foUowing remark. The (generahzed) Hall polyno- 
mials are known to have leading coefficient equal to one. Hence, by the Lang- Weil 
theorem, all the varieties EnHOm have a unique irreducible component of maximal 
dimension. 

4. The preprojective variety and coefficients of the bar 

automorphism. 

4.1. For any arrow a : i ^ j in the quiver Q, we define i{a) = i and h{a) = j . Let 
Q°P be the opposite quiver, having the same vertices as Q, and an arrow a* : j —>■ i 
for each arrow a : i — > j in Q. 

Fix a dimension type d = J2i "^i* i^ -f*^-^- ^^ above, Gd acts on Rd{Q) = Rd and 
on Rd{Q°^). Note that the map sending a linear map to its adjoint induces an 
isomorphim RdiQ) -^ i?d(Q°^), M ^ M*. 

Let Hd be the preprojective variety (see fSl): 

Hd := {((Afa)o, (A^a-)c.) e RdiQ) X RdiQ^n, for alH e / : 

J2 N^'M^= Y. Ma,Na,.}GRdiQ)xRd{Q°P). 

a£Qi,h{a) = i a£Qi,i{a)=i 

We denote by p (resp. p°P) the canonical projection H^ -^ Rd{Q) (resp. H^ -^ 
Rd{Q°^))- We also define a twisted projection p : H^ — >• Rd{Q) by mapping (M, TV) 
to M + N* using the identification Rd{Q) — Rd{Q°^) above. 

4.2. Observe that, for all N in Rd, the identification Rd{Q) -^ Rd{Q°^) maps the 
orbit Gd-N to Gd-N* and the tangent space Tn{On) to Tn'{On')- 

We consider the non degenerate pairing on Rd{Q) x Rd{Q°^) given by 

< (Af„),(A^„.) >= ^ Tr{M^N^,). 

We have the following 

Lemma 4.1. Wif/i respect to the pairing above, we have Tn{On)'^ = P°^{p^^{N)), 
for all N in RdiQ). 

Proof. By LemmalTlfin. an element (/„.) oi Rd{Q°P) belongs to Tn{On)-^ if and 
only if for all X in Qd, we have 

Y, Tr({Xn(^)N^ - iV„X,(„))/„0 = 0. 

By well known properties of the trace form, this is equivalent to 
Y Tr{X^^)N^f^,) - Tr{X,^^)f^,N^) = 0, 

aGQi 

thus 

YM Y ^o^fa' - Y ^"•^") = ^- 
i£l i{a) — i h{a)—i 

Since the trace form is non-degenerate, this gives 

J2 ^-U' - Y. •^"*^" = 0, 

z(a) — i /i(q)— -i 

which proves (/a*) ^ p°^(7?~^(A^)) as required. D 
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For any morphism tt : X ^ Y oi fc-varieties, and any function / (resp. g) on X 
(resp. Y), we define as usual : 

7r*(5) ■■ X ^ k, X ^ g{7r{x)), 

Tt(x)=y 

For a subset A of Rd or Rd{Q°^), we denote by 1a the corresponding characteristic 
function. The previous lemma gives the following interpretation of the polynomials 
^M,N in terms of the geometry of the preprojective variety: 

Proposition 4.2. For M in Rd{Q) and N* in Rd{Q°^), we have 



{p''n*{pn^ou){N*)^^M,N. 

Proof. Obviously, (p°p)*(p)*(1om) belongs to CgAMQ"")]- 

Fix N in Rd- By the definitions, (p°p)*(p)*(1om)(^*) can be rewritten as 
{p°^)*Um){N*), where the function / on Wd is defined by fM{A,B*) = 1 if 
j4 + _B G Om , and otherwise. Hence, by the previous lemma, 

(p°^)*(p)*(1om)(^*) = E fM{A,N*) = \OMfMN + TM.{ON')^)\. 



By Theorem rOl this gives {p°p)^{P)*{Iom){N*) = ^m,n. □ 
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